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Abstract. - The sum of the average work dissipated plus the information gained during a ther- 
modynamic process with discrete feedback must exceed zero. We demonstrate that the minimum 
value of zero is attained only by feedback-reversible processes that are indistinguishable from their 
time-reversal, thereby extending the notion of thermodynamic reversibility to feedback processes. 
In addition, we prove that in every realization of a feedback-reversible process the sum of the work 
dissipated and change in uncertainty is zero. 
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Investigations into the thermodynamic implications of 
feedback have a long history |1I2| , especially with regard to 
the relationship between information acquisition and ther- 
modynamic quantities - such as work, heat, and entropy. 
Still, there is no complete theoretical framework detailing 
the relationship between feedback and thermodynamics. 
Interest in developing such a framework - a thermody- 
namics of feedback - has grown recently in part due to ex- 
periments on feedback cooling [3J[3] and feedback control 
of nanoparticles j5,:6j; experimental, computational, and 
theoretical studies of feedback driven Brownian ratchets 
[7H12]: and new theoretical predictions relating dissipa- 
tion to information [13H23] . 

Recently, Sagawa and Ueda derived a generalization of 
the second law of thermodynamics for quantum and clas- 
sical systems manipulated by one feedback loop p~8 | H9 J . 
which subsequently has been verified experimentally [24] 
and extended to classical systems driven by repeated dis- 
crete feedback - implemented through a series of feed- 
back loops initiated at predetermined times - indepen- 
dently by Horowitz and Vaikuntanathan [2D], and Fujitani 
and Suzuki |21) . The second law of thermodynamics for 
discrete feedback states that the average work dissipated 
(Wd) in driving a system with a discrete feedback proto- 
col from one equilibrium state at inverse temperature /? 
to another at the same temperature is related to the mi- 
croscopic information gained through measurements (I) 

by 

P(W d ) + {I) > 0. (1) 
Here, (I) is the mutual information between the state 



of the system and the measurement outcome [T8HT51l2"5] ■ 
and the average work dissipated is the average work (W) 
done in excess of the average free-energy difference ( AF) : 
(Wd) = (W) — (AF). The nomenclature derives from 
the observation that in the absence of feedback [(I) — 0], 
eq. ((T|) reduces to a statement of the second law of ther- 
modynamics: (Wd) > 0. 

The second law of the thermodynamics plays a central 
role within the framework of thermodynamics. Besides 
restricting the realizability of thermodynamic processes, 
it distinguishes particular processes that produce no en- 
tropy. In macroscopic systems, such processes are called 
reversible, because the sequence of equilibrium states vis- 
ited by the system during the process can be traversed 
both forwards and backwards [26 ]. At the microscopic 
level the connection between entropy production and re- 
versibility must be interpreted statistically and is quanti- 
fied mathematically by the distingiushibility of a thermo- 
dynamic process from its time- reversal [2211231127] . Like the 
second law of thermodynamics, eq. (flj singles out partic- 
ular processes that have no dissipation [(3(Wd) + (/) = 0]. 
Because such processes are important in thermodynam- 
ics, it is of value to characterize them in the presence of 
feedback; especially since they are optimal processes that 
most efficiently convert information into work. Thus, in 
this letter we analyze those processes that saturate the 
bound in eq. (P) [/3(Wd) + (I) = 0], and we demonstrate 
that such processes satisfy a new, distinct criterion, simi- 
lar to thermodynamic reversibility, which we call feedback 
reversibility. Intuition from the second law of thermody- 
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Fig. 1: Illustration of the feedback protocol driving a particle 
(dot) that makes thermally activated transitions between two 
states L and R. From left to right is the evolution of the 
relative heights of the energy levels of the two states (horizontal 
bars) at four times, t — — oo,0, + and oo, during the protocol 
A R = {\ R }^L associated to measuring the particle to be in 
state R at t — 0. Above each state is the probability to be in 
that state conditioned on implementing protocol A R , p t (k\A R ), 
k — L,R. 



namics might lead one to naively expect that the bound 
in eq. ([IJ can always be reached as long as the process 
is sufficiently slow; however, this is generally not true, as 
we will demonstrate. To saturate eq. ([1]) one must bal- 
ance (Wd) and (I), which requires adjusting the driving 
protocol, the type of measurement, and the measurement 
error. 

Before considering the most general situation, let us first 
analyze a simple toy model of feedback, inspired by the 
Szilard engine [TJ|2]. We will demonstrate that by adjust- 
ing the measurement error we can saturate eq. ([T]) even 
when the process is not adiabatically slow. Consider a 
particle weakly coupled to a thermal bath at inverse tem- 
perature (3 = 1 that makes thermally activated jumps be- 
tween two states, labeled left (L) and right (R). The state 
energies Ei{\), i = L,R, depend on a vector of external 
parameters A which we vary using feedback during a time 
interval t e (—00,00). Initially at t = — 00 the particle 
is in equilibrium with the energies equal. From t = — 00 
to 0, the system evolves freely. At t = 0, we measure 
the particle's state, misidentifying it with error e, i.e., the 
probability to measure the particle in R (L) given it is in 
L (R) is P(R\L) = e [P(L\R) = e}. Immediately after the 
measurement at t — + , we vary the external parameters 
according to a protocol that depends on the measurement 
outcome: if the particle is measured to be in state R (L), 
we initiate protocol A R = {Af (A L ), depicted in fig.Q] 
by instantaneously lowering the right (left) energy level 
by AV/2, while simultaneously raising the left (right) en- 
ergy level by AV/2; so that the energy difference between 
the two states is AV. Finally, from t = + to 00, we 
quasi-statically return the energy levels to their original 
values. (Similar protocols were utilized to study dissipa- 
tion in nonselective quantum measurement [28] and the 
achievability of the equality in eq. ([1} for quantum feed- 
back given a particular measurement protocol [29].) 

Figure [T] contains a depiction of a realization of this pro- 
cess where the protocol A R is executed. At the four times 
t = —00, 0, + , and 00 along the process, we depict the rel- 
ative heights of the energy levels of the two states. Above 
each energy level is the probability at time t to be in that 



state conditioned on implementing protocol A R , p t (k\A R ), 
k = L, R. Initially the particle is in equilibrium with en- 
ergy levels equal, therefore p- 00 (R\A R ) = p-oe^A 11 ) = 
1/2. At t = 0, we measure the particle to be in state 
R; consequently, po(R\A R ) = 1 — s and po(L\A R ) = e re- 
flecting that with probability e protocol A R is mistakenly 
implemented when the particle is in state L. Immediately 
after the measurement we instantaneously change the en- 
ergy levels. Since this step is infinitely fast, the conditional 
probability distributions do not vary. Finally, starting at 
t = + we infinitely slowly return the energy levels to 
their original configuration, so that at t = 00 the system 
has returned to its initial equilibrium. 

To explore how eq. (JTJ depends on the error e, we deter- 
mine the values of (Wd) and (I) as functions of e. First, 
observe that since each protocol is cyclic the average free- 
energy difference is zero (AF) — 0, and the dissipated 
work equals the work, (Wd) = (W). Furthermore, the 
symmetry of A R and A L implies that the average work 
during each protocol is the same. Thus, we focus on the 
average work during A , which we calculate in two steps. 
First, we compute the average work conditioned on imple- 
menting protocol A^ during the instantaneous switching 
of the energy levels at t = + : 



I \ AV 



(2) 



Second, from time t = + to 00 the process is quasi-static. 
Therefore, the average work given A R during this period is 
the free-energy difference between the equilibrium states 
when the energy levels differ by and when the energy 
levels are equal: 
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Noting that each protocol, A R and A L , occurs with equal 
probability and that the work during each protocol is the 
same, we conclude that the total average work equals the 
sum of eqs. ([2]) and ([3]), which, after some algebraic ma- 
nipulation, can be expressed as 
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The mutual information (I) [see eq. (fTTj) below] quantifies 
the reduction in our uncertainty about the microscopic 
state of the system upon making a measurement. It is 
defined as the relative entropy between the joint proba- 
bility distribution of the state of the system k = L, R at 
the time of measurement (t — 0) and the measurement 
outcome m = L, R, 
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with the product of their respective marginal distribu- 
tions, p (k) = 1/2 and P{m) = 1/2 [5S]. Therefore, the 
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mutual information reads 
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In fig.[2]we plot (W) [eq. flU], (I) [eq. ©], and their sum 



(W) + <I) = (l-e) In 



(l- e )(l + e^) 



:ln [e(l + e Ay )] 
(7) 

as functions of the error e. For error-free measurements 




Fig. 2: Plot of {W) (dashed), (/) (long dashed), and their sum 
(W) + (I) (solid) as a function of the measurement error e for 
AF = 2. 

(e = 0), we are able to extract the maximum amount of 
work [— (W) is largest] and obtain the maximum amount 
of information (/) = In 2. However, (W) + (I) > 0; some 
of the information is not used to extract work. At the ex- 
pense of increasing e, decreasing the amount of work ex- 
tracted, and decreasing the amount of information gained, 
we can reach the bound in eq. ([1]). From eq. (JT]), we see 
that the sum (W) + (I) is zero when e equals 
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Thus, by adjusting the measurement error with fixed 
external parameter protocols we can construct a feed- 
back protocol that is not adiabatically slow and satisfies 
(i{W d ) + (I) = 0. 

A physical interpretation can be given to £o in eq. © 
by considering the time-reversal of the preceding feedback 
process. The time-reversal of a feedback process is not 
trivial, since there is no such thing as the time-reversal 
of a measurement. However, we follow Ref. [20] and con- 
struct a distinct thermodynamic process termed the re- 
verse process, which will act as the time-reversed feed- 
back process. The reverse process begins by randomly 



selecting a protocol, A L or A^, according to the proba- 
bility that the respective measurement outcome, L or R, 
occurs during feedback: P(L) = P(R) = 1/2. The par- 
ticle is then driven away from equilibrium by using the 
time-reversal of the selected protocol, A fc = {A A i t }^_ 00 , 
k = L, R. Now, imagine randomly selecting protocol 
A R , and consider the evolution of the system conditioned 
on executing the time- reversed protocol A R , depicted in 
fig. [3] Initially the particle is in equilibrium with ener- 
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Fig. 3: Illustration of the reverse process at four times, t = 
— oo, (F, and oo (right to left), along the protocol A R . Above 
each state is the conditional probability distribution p t (k\A R ), 
k — L,R. 



gies equal. As a result, the probabilities to find the par- 
ticle in states L and R at t = — oo in the reverse pro- 
cess conditioned on the protocol A R are initially equal: 
jO-oo^A^) = p-oo(L\A R ) = 1/2. From t = -oo to 
the right energy level is quasi-statically lowered by 
AV/2 while the left energy level is raised by AV/2. Thus, 
at t = 0~ the conditional probability distribution is a 
Boltzmann distribution: p -(L\A R ) = 1/(1 + e AV ) and 
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At t = 0, the energy levels 



are instantaneously returned to their original equal values; 
the conditional probablity distribution does not change. 
Finally, from t — to oo the energy levels are held fixed, 
while the system relaxes back to its initial equilibrium. 

Comparing figs. Q] and El we see that when e = e 
[eq. ((8])] the conditional probability distributions in the 
feedback and the reverse processes are equal at the main 
stages of the process: 



p t (k\A R ) = p_ t (k\A R ), 



(9) 



for k = L,R. The same holds for A L . Below we demon- 
strate that this equality holds at any time t [see eq. (|18[)]. 
Moreover, since each protocol is implemented with equal 
likelihood in both the original feedback process and the 
reverse process, the joint distributions of states and pro- 
tocols for the feedback and reverse processes are equal 
for an optimal feedback process: p t (k,A m ) = p_ t (/c, A m ), 
k, m = L,R. Notice that the reverse and feedback pro- 
cesses are of a different nature: there are no measurements 
in the reverse process. Despite this difference, identify- 
ing the reverse processes as the time-reversed feedback 
processes is consistent with the thermodynamic principle 
linking reversibility and dissipation: the process with least 
dissipation - the protocol which saturates eq. ([T]) - cor- 
responds to the situation where the feedback process is 
indistinguishable from its time-reversal (from the reverse 
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process). Consequently, we call these processes feedback 
reversible or simply reversible. 

We now demonstrate that this conclusion holds quite 
generally as a consequence of the detailed fluctuation the- 
orem for discrete feedback [eq. (fP3"]) below] [3D]. This the- 
orem relates the fluctuations in two thermodynamic pro- 
cesses related by time-reversal: the forward and the re- 
verse processes. Our analysis begins by sketching a deriva- 
tion of eq. ([T]) based on the detailed fluctuation theorem 
for discrete feedback originally presented in Ref . [20] . For 
clarity of exposition, we limit our discussion to feedback 
processes with only one feedback loop initiated at the be- 
ginning of the process. All our conclusions can be gener- 
alized to situations with repeated discrete feedback. 

Consider a classical system, whose position in phase 
space (or microscopic configuration) at time t is z t . This 
system is driven away from equilibrium at inverse tem- 
perature (3 from t — to r using one feedback loop as 
follows: at t — a physical observable M is measured 
with outcomes m that occur with probability P(m\zo), 
conditioned on the system's state at the time of measure- 
ment zq. Based on the outcome of this measurement, the 
system is driven by varying a set of external parameters A 
with time using the protocol A m = {A™}[ =0 from A™ = A 
to A™ = B m . Finally, at time t = r the external parame- 
ters are held fixed at A™ = B m while the system relaxes 
back to equilibrium at inverse temperature /3. Repeatedly 
executing this sequence of actions - each time equilibrat- 
ing the system, driving the system using feedback, and 
then re-equilibrating the system - generates an ensemble 
of realizations of the forward process. In each realiza- 
tion, thermal fluctuations will cause the system to trace 
out a different microscopic trajectory through phase space 
7 = {zt}J— . The joint probability to observe 7 with A m 
is V[y; A" 1 ]. The work dissipated during this realization, 



equal to the probability to implement the forward protocol 



Wd[7; A m ] = W[r,A m ] ~ AF[A m ], 



(10) 



is the work W[y; A m ] done in excess of the free-energy dif- 
ference AF[A m ]. Here, AF[A m ] depends on the executed 
protocol, because the final external parameter value is a 
function of the measurement, A™ = B m . Moreover, mea- 
surements made upon initiating the feedback loop change 
our uncertainty about the microscopic state of the system 
by an amount [20 



/[ 7 ;A m ]=ln 



P(m\z ) 



P{m) 



(11) 



where P{m) is the (unconditional) probability to observe 
outcome m when measuring the physical observable M. 
The average of I over many realizations is the mutual in- 
formation that appears in eq. (JlJ . 

The reverse process is defined as in our previous exam- 
ple. In the reverse process no measurements are made. 
Instead, we drive the system away from equilibrium using 
a reverse protocol A™ = {A™}[ =0 with A™ = A™_ 4 , which 
is selected randomly with probability 7r[A m ] defined to be 



A™ = A™_ ( in the forward process 7r[A m ]: 



7f [A m ] = 7r[A m ] = J d-yV[r,A m }, 



(12) 



where d~y is a measure on microscopic trajectory space. Af- 
ter selecting A" 1 , the system is equilibrated with a thermal 
bath at inverse temperature /3 with external parameters 
fixed at A™ = A™ = B m . Observe that the initial equi- 
librium distribution of the reverse process corresponds to 
external parameter value B m and depends on which proto- 
col A™ is implemented [2UJ. From t — to r the external 
parameters are varied according to the reverse protocol 
A m = {A™}[ =0 . At t = r, the external parameters are 
fixed to A™ = Xq = A while the system relaxes back to 
equilibrium. For every microscopic trajectory of the for- 
ward process 7 = {z t }l =0 , there is a conjugate reverse 
trajectory 7 = {zt}l =0 , where z t = z*_ t and z* denotes 
momentum reversal. The probability to observe reverse 
trajectory 7 and reverse protocol A m in the reverse pro- 
cess is v\y; A™ 1 ]. 

Having introduced notation and presented the defini- 
tions of the forward and reverse processes, we now state 
the detailed fluctuation theorem [20] : 



V[f,A ri 



/3W d [7;A m ]+/[7;A m ] 



(13) 



Because no measurements are made in the reverse process, 
there are microscopic trajectories and reverse protocols 
that occur together in the reverse process whose conjugate 
microscopic trajectories and conjugate protocols do not 
occur together in the forward process; it is possible for 
V ^ when V = [20] . 

Equation (|13|) implies that the relative entropy, 
D(f\\g) = Jdxf(x)hi(f(x)/g(x)), between V and V 
is (201 



D(T\\V)=f3(W d ) + {I), 



(14) 



where (•) is an ensemble average over realizations of the 
forward process. D(V\\V) measures the distinguishability 
of the forward and reverse processes; it is a microscopic 
measure of the intensity of the "arrow of time" (22][23] . 

Equation now follows by exploiting the nonnegativ- 
ity of the relative entropy (D > 0) [25] in eq. ([14]) . More- 
over, eq. (|14[) implies that thermodynamic processes for 
which f3(Wd) + (-0=0 are those with D = 0, which is 
true if and only if 



V[ T ,A m ]^P[f,A r ' 



(15) 



for all 7 and A m [25]. The condition D = additionally 
requires that the supports of V and V - the sets of micro- 
scopic trajectories and protocols for which V and V are 
nonzero - must be identical: every microscopic trajectory 
and reverse protocol that occur together in the reverse 
process have conjugate pairs that occur together in the 
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forward process. Equation (|15p is a microscopic state- 
ment of reversibility: the process looks the same forwards 
and backwards in time, since every realization occurs with 
equal likelihood in the forward and reverse processes. We 
conclude that the inequality in eq. ([TJ is saturated only 
when eq. (|15p is satisfied, that is only for reversible pro- 
cesses. 

One could in principle consider a "super-system" com- 
posed of our system of interest and a feedback mechanism 
- formed from a controller that manipulates the parame- 
ters A and a memory that records the measurement out- 
comes. While the explicit implementation of such a set-up 
with a time-dependent Hamiltonian is a difficult task, it is 
still instructive to assume that a super-system exists and 
to compare the feedback reversibility we have introduced 
with standard thermodynamic reversibility. In this set-up, 
we manipulate the super-system by varying a collection of 
parameters external to both the system of interest and the 
feedback mechanism according to a predetermined cyclic 
protocol. The work performed by the external parameters 
along this cycle is responsible for recording the measure- 
ment of the system, the control in response to the mea- 
surement, and the erasure of the measurement. As shown 
by Sagawa and Ueda [16| . under rather general hypothe- 
sis, the work to measure (W mcas ) plus the work to erase 
(W eras > is bound by /3«W eras ) + (W mcas )) > </}. Combin- 
ing this bound with equation ([1]), we find that the total 
work performed on the super-system is 

(Wd) + (W cras ) + (W mcas > > 0, (16) 

which is the second law of thermodynamics applied to 
the super-system when the feedback mechanism has gone 
through a cycle. To achieve the equality in equation (fT6l) . 
and consequently in ([I]), one would need thermodynamic 
reversibility in the full super-system - that is the thermo- 
dynamic process must be indistinguishable from its time- 
reversal in the phase space of the super-system. However, 
such a process is not quasi-static, because during the writ- 
ing and erasure of the memory the super-system is not in 
equilibrium. We have shown that to achieve the optimal 
feedback control, equality in ([T]), one only needs feedback 
reversibility in the system, while the controller can be out 
of equilibrium. 

From eq. (|T5|) follows another useful microscopic state- 
ment of reversibility in terms of the phase space densities 
conditioned on the executed protocol [eq. (fT8| below] . Re- 
call that in our construction of the reverse process, the 
probability to execute A™ in the forward process 7r[A m ] 
equals the probability to use the conjugate reverse pro- 
tocol A m in the reverse process 7r[A m ] [eq. (fT2|) ]. Thus 
dividing both sides of eq. (fl~5j) by n[A m ] — 7r[A m ] we find 
that, for a reversible process, the statistics of the trajec- 
tories conditioned on the protocol executed in the forward 
and reverse processes are also indistinguishable: 

Vh\A m ] = V[j\A m ]. (17) 



Furthermore, by integrating these conditional trajectory 
distributions over all trajectories that pass through z t — 
we find that the conditional phase space densities 
must also be equal: 

p(z t \A m ) = p(z T _ t \A m ), (18) 

which is the general expression corresponding to eq. ©• 

Equation (|18j) is valid at all times during a feedback- 
reversible process. In particular, it implies that in our 
preceding example eq. ([9]) is true even during the time 
interval t £ (— oo,0) prior to the measurement. The re- 
versibility of the process during this interval may be sur- 
prising at first, since during this period the system is freely 
evolving with the external parameters held fixed. To gain 
further insight, consider a special case of our example with 
no measurement error, e = 0. In the forward process 
during the interval t 6 (— oo,0), p t {k\A R ) represents the 
evolution of the (probabilistic) state of the system con- 
ditioned on the future measurement outcome being R at 
t = 0. Whereas during this interval, p-t(k\A R ) is a relax- 
ation from a non-equilibrium distribution ^(fclA^) = 5k, r 
to equilibrium /^(fclA^) = 1/2. The reversibility condi- 
tion eq. (fT8|) [eq. ((9J] states that these two processes are 
identical upon time reversal. In fig. 01 we illustrate this 
reversibility by depicting the evolution of an ensemble of 
systems during t £ (— oo,0). The ensemble is in equilib- 
rium from t = — oo to 0, but the sub-ensemble of systems 
that are in the right (left) state at the time of measurement 
t = exhibits an evolution that is identical to the time- 
reversal of the corresponding relaxation. The underlying 
reason is microscopic reversibility: the probability in equi- 
librium to observe a microscopic trajectory and its time 
reversal are equal [30] ■ As a consequence, the probability 
to observe a microscopic trajectory conditioned on being 
at a particular state in the future, say z' , is the same as 
the probability to observe the time reversal of this trajec- 
tory while the system relaxes to equilibrium when initially 
at z' . 

Finally, we note that in every realization of a reversible 
process (3Wd + 1 = 0, which follows by substituting 
the reversibility condition eq. (|15p into the nonequilib- 
rium detailed fluctuation theorem [eq. (fT3|) ]. Although 
/3Wd + 1 = for every realization of a reversible process, 
the value of Wd (J) may differ in each realization. 

We have demonstrated that those feedback processes 
that most efficiently use information to extract work [that 
saturate eq. ([1])] are feedback reversible - they are in- 
distinguishable from their time-reversal, thereby extend- 
ing the concept of thermodynamic reversibility to feed- 
back processes. Like reversible thermodynamic processes, 
feedback-reversible processes are ideal for experimentally 
measuring free-energy differences. When all external pa- 
rameter protocols end at the same value A™ = B - in- 
dependent of m - the sum of the work and information 
along a feedback-reversible process equals the free-energy 
difference. Feedback-reversible processes can also be used 
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Fig. 4: During the time interval t — — oo to 0, the external 
parameters are constant with time; therefore, an ensemble of 
systems initially in equilibrium remains in equilibrium. How- 
ever, the sub-ensemble of systems that at t — are measured 
to be in the right (left) state - depicted here as the four upper 
(lower) copies of the system - does vary in time. The proba- 
bility in this sub-ensemble to be in state L at time t, pt (L\A R ) 
[p t (L[A L )], is plotted as a function of time during the interval 
t 6 ( — oo,0), assuming error-free measurement (e = 0). In a 
reversible process, the time reversal of this evolution is identi- 
cal to a relaxation to equilibrium from a nonequilibrium state 
given by p (L\A R ) = [ P o(L\A L ) = 1]. 



to estimate the information gain by measuring the work 
dissipated. In our illustrative example, if we were to mea- 
sure the work as as function of AV its minimum value 
would equal (I) [eq. In general, a measurement of 

the work disspated gives a lower bound on the informa- 
tion according to eq. ([T]) . Furthermore, our work gives an 
indication on how to design feedback-reversible protocols 
for a given measurement scheme, which may be a difficult 
task [3lJ[32]- The protocols must be such that eq. (fl8|) 
holds at every instant. In particular, immediately after 
a measurement with outcome m, the conditioned phase 
space density must be identical to the conditioned density 
prepared by the protocol A™ run in reverse. This identity 
is the key ingredient for designing an optimal protocol. 
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